Automatic Extraction of Textured Vertical Facades from Pose Imagery

Extracting 3-dimensional structure from real-world imagery and rendering it from unrestricted viewpoints is an important problem in computer vision, and increasingly, computer graphics. Despite many years of research, a system that automatically recovers realistic 3-D models from images remains elusive; most practical systems require signi cant human input. However, unlike automatic algorithms, human-assisted systems are not scalable, both in terms of the number of images and the complexity of the model being reconstructed. This paper describes an automatic 3-D model extraction algorithm based on the following ideas: Each image in our input dataset is annotated with accurate estimates of camera position and orientation (pose), to become a fundamentally more powerful datum, a pose image. Pose information is used in our algorithm to constrain the reconstruction process, and to focus on processing only a portion of the dataset that is relevant to a given 3-D region. We exploit geometric structure inherent in typical urban environments. In particular, we focus on vertical facades, as they are common in urban scenes. The vertical facade extraction algorithm detects likely facade azimuths using image-space information of horizontal line segments, and locates them using a space-sweep algorithm. We exploit the availability of a large set of observations of each facade to design a simple texture estimation algorithm that is statistically robust with respect to illumination changes and occlusion. We present results of the algorithm for a large pose image dataset (consisting of about four thousand images taken from eighty-one positions) of an urban o ce complex. Our algorithm was successful in recovering all signi cant vertical facades in the complex, as well as several neighboring facades.